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Filyokov and Karpov [Inzhenerno-Fizicheskii Zhurnal 13, 624 (1967)] have proposed a theory of 
non-equilibrium steady states in direct analogy with the theory of equilibrium states : the princi- 
ple is to maximize the Shannon entropy associated with the probability distribution of dynamical 
trajectories in the presence of constraints, including the macroscopic current of interest, via the 
method of Lagrange multipliers. This maximization leads directly to generalized Gibbs distribution 
for the probability distribution of dynamical trajectories, and to some fluctuation relation of the inte- 
grated current. The simplest stochastic dynamics where these ideas can be applied are discrete-time 
Markov chains, denned by transition probabilities Wi^j between configurations i and j : instead of 
choosing the dynamical rules Wi^j a priori, one determines the transition probabilities and the as- 
sociate stationary state that maximize the entropy of dynamical trajectories with the other physical 
constraints that one wishes to impose. We give a self-contained and unified presentation of this type 
of approach, both for discrete-time Markov Chains and for continuous-time Master Equations. The 
obtained results are in full agreement with the Bayesian approach introduced by Evans [Phys. Rev. 
Lett. 92, 150601 (2004)] under the name 'Non-equilibrium Counterpart to detailed balance', and 
with the 'invariant quantities' derived by Baule and Evans [Phys. Rev. Lett. 101, 240601 (2008)], 
but provide a slightly different perspective via the formulation in terms of an eigenvalue problem. 



In contrast to the statistical physics of equilibrium, which is based on a few general principles, everybody complains 
that no global theory has emerged to describe out-of-equilibrium systems. A natural explanation of this matter 
of fact is that the problem is ill-defined for many reasons. A first reason is that there are actually very different 
ways to be out-of-equilibrium : a system can be 'out-of-equilibrium' because it has a very slow dynamics that never 
converges towards some stationary state, or because it is in a non-equilibrium steady state, characterized by some 
currents that are imposed by the environment. It is clear that in the first case, one really needs a non-trivial 
dynamical description, whereas in the second case, one may hope that the 'steady state' property could help to 
develop some theory generalizing the equilibrium steady state. A second reason is that even within the field of 
'non-equilibrium steady states', very different goals have been pursued. On one hand, many theorists of statistical 
physics consider that the best strategy consists in solving specific models with chosen dynamical rules, in order to 
see if some general principles emerge in the end : in this direction, the most studied models are one dimensional 
exclusion processes, for which many remarkable results have been obtained over the years (see the recent short 
review [l| and references therein). On the other hand, it is clear that the success of the theory of equilibrium is 
based on the fact that one does not specify the details of the dynamical evolution but that on the contrary, one 
keeps only its conservation properties (for instance the energy conservation) and then maximize the Shannon entropy 
associated with the equilibrium state. So there has been various attempts to formulate general principles based on 
some notion of 'entropy' for non-equilibrium steady states. Although the principles of 'minimum entropy production' 
and of 'maximum entropy production' seem popular, in particular in applications to related areas like chemistry, 
biology or climatology, the proposed justifications have not been widely accepted, either because there are at the 
level of 'generalized thermodynamics' rather than statistical physics, or because they are limited to 'near-equilibrium' 
conditions, or because they involve other additional questionable assumptions (see for instance the discussions in 
and references therein). But we feel that instead of focusing on the notion of 'entropy production' to formulate general 
principles, it is much more natural to consider the 'Shannon entropy associated with the probability distribution 
of dynamical trajectories' that we will call equivalently 'dynamical entropy' to have a shorter name. This notion 
can be introduced for any stochastic dynamics, or for any deterministic chaotic dynamics that becomes effectively 
'stochastic', as follows : to characterize the dynamics during some time interval [0,t], one considers all possible 
dynamical trajectories f2[o,t] and their probabilities P(fi[o,t]) normalized to 
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The Shannon entropy associated with this distribution V(fl[o^]) over dynamical trajectories reads 

S dyn (t) = - ^P(O [0it] )lnP(O [0)t] ) (2) 



i[0,t] 



For many stochastic process of interest, this entropy becomes extensive in the time t in the large-time limit t — > +00 : 
it is then convenient to introduced the entropy rate per unit time, called the Kolmogorov- Sinai entropy in the context 
of chaotic dynamical systems (see for instance the book [6| and references therein) 

h KS = lim (3) 

t— s-+oo t 

More than forty years ago, Filyokov and Karpov 0-0 have proposed to base the theory of non-equilibrium steady 
states on the maximization of the dynamical entropy of Eq. [2] in the presence of some constraints via the method of 
Lag range multipliers. We are not aware of direct continuations of these works, except the very recent work of Favretti 
[lOj |. However closely related ideas have been proposed independently by Evans [TTj under the name 'Non-equilibrium 
Counterpart to detailed balance' and have been applied to shear flow |12Hl4| . Also among recent attempts to 'prove' 



the principle of maximum entropy production by Dewar 151 ] ■ one can see that the starting point is precisely the 



maximization of the dynamical entropy of Eq. [2j We thus feel that if one wishes to formulate a general principle 
based on the maximization of some notion of entropy for non-equilibrium steady states, the dynamical entropy of Eq. 
is definitely the most natural and could be the basis of some consensus. 

Let us now mention various recent works that are related to some of these notions : 

- For random walks on arbitrary networks, the optimization of the dynamical entropy of Eq. pleads to the 'maximal 
entropy random walk' that has been introduced by Burda, Duda, Luck and Waclaw [l6| : it is different from the usual 
random walk that maximizes the entropy only locally whenever there exists some fluctuations in the coordination 
numbers of nodes of the network (but of course these two types of random walks coincide on regular networks). 

- For non-equilibrium steady states, even within the point of view where the dynamical rules are given a priori, 
the dynamical entropy of Eq. [5] plays a major role in the 'thermodynamics of histories' that has been developed by 
Lecomte, Appert-Rolland and Van Wijland [l7|, and that has been then applied to various glassy systems Il8ll. 

- The method of Lagrange multipliers has been much used recently for non-equilibrium quantum systems [19I422T ] , 
but we should stress that the physical meaning is completely different : in these quantum studies, the dynamics is 
governed by some given quantum Hamiltonian, and the Lagrange multiplier associated with a macroscopic current 
is added to the Hamiltonian, whereas in the Filyokov-Karpov approach [7H10l| or in the Evans approach [Tl| - [l4| the 
dynamical rules are not given at the beginning but on the contrary they are determined by the optimization of the 
dynamical entropy of Eq. [5] in the presence of appropriate constraints. 

The aim of this paper is to give a self-contained and unified presentation of this type of approach, to discuss 
the various results that can be obtained, and to make the links with other recent developments in the field of non- 
equilibrium statistical physics. 

The paper is organized as follows. In section UH we describe how the maximization of the dynamical entropy of 
Eq. pleads to generalized Gibbs distribution for the probability distribution "P^jo,*]) of dynamical trajectories, and 
to some fluctuation relation of the integrated current. In the remainder of the paper, we discuss in details how this 
general formal argument can be given a more precise meaning when one considers well defined stationary stochastic 
dynamics. In section lHTl we recall the important properties of discrete-time Markov chains, that constitute the simplest 
framework to describe stochastic dynamics with a time-extensive dynamical entropy (Eqs [5] and [3]) . In section IIV1 
we explain how the maximization of the dynamical entropy of Eq. [5] in the presence of a constraint concerning the 
energy of dynamical trajectories allows to recover the usual Boltzmann-Gibbs distribution of equilibrium. In section 
fVl we add a supplementary constraint concerning the flux of some observable and derive the form of the optimal 
solution for the stationary distribution and for the transition rates. We then focus on the problem of finding effective 
local stochastic rules, and explain how a consistent formulation can be obtained by the maximization of the relative 
dynamical entropy with respect to the equilibrium trajectories : we describe the case of discrete-time Markov Chains 
in section IVT1 and the case of continuous-time Master Equations in section IVTTl In both cases, we make the link with 
the Evans approach called 'Non-equilibrium Counterpart to detailed balance' Our conclusions are summarized 

in section IVIIII Appendix [X] contains a reminder on the maximization of static entropy for the equilibrium, to make 
the comparison with the maximization of the dynamical entropy discussed in the text. In Appendix [B] we explain 
the technical simplifications that occur if one considers an alternate Markov chain as done originally |7H10|| . 
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II. GENERAL IDEA : GENERALIZED GIBBS DISTRIBUTION FOR DYNAMICAL TRAJECTORIES 



As recalled in Appendix lAl the usual Boltzmann-Gibbs distribution for equilibrium configurations can be obtained 
from the maximization of the static entropy in the presence of constraints introduced via Lagrange multipliers. This 
simple derivation of the equilibrium has been introduced by Jaynes [23| , and can be found nowadays in most textbooks 
on statistical ph ysic s. Although Jaynes has often proposed to apply the same strategy to non-equilibrium phenomena, 
in particular in |24j , we have not been able to find in his articles the expression of Eq. [5] for the dynamical entropy, 
whereas Eq. [2] is the clear starting point of some 'maximization' in the works of Filyokov and Karpov and in the 
Evans approach [Tl| - [T^ . (Note that Eq. [2] is also the starting point in the works of Dewar [15j . but with the different 
goal of justifying the 'maximum production entropy principle'). 



A. Maximization of the dynamical entropy with constraints 



For non-equilibrium steady states characterized by some macroscopic current Jo, the general idea is the following 
one wishes to optimize the dynamical entropy of Eq. [5] with constraints concerning the normalization of Eq. [1] 



Af= ^P(fi [0)t] ) = l (4) 

£2[o,t] 



the averaged energy of the trajectory 



Edyn{t) = V(Q m )E(Sl m ) = tEo (5) 



and the averaged current of the trajectory 



Jd yn {t) ^ £ P(O [0>t] )J(O [0i{] ) = tJ (6) 

f2[o,t] 



The optimization of the Lagrange functional 

if, = S dyn (t) - p (AT - 1) - (E dyn {t) - tE ) + v ( J dyn (t) - t J ) 

= - P(%t]) ln^(fi[o, t] ) - P I E Wo,*]) - 1 

^[0,t] \^[o,t] 



with respect to P(£l[ ,t]) 



-p [ E H^[o,t)) E ( n [o,t]) - tE + v E r(n m )j(n [0A ) - tj ] (7) 

V,t] / \ n [o.t] 



(5* 

= ^77i V =-lnV(n [0tt] )-l- p-/3E(n [0tt] ) + vJ(Sl m ) (8) 



directly leads to the following generalized Gibbs distribution for dynamical trajectories 

V(Cl m ) = e -i-p-^("[o, t ])+^(f2[o, tl ) (g) 

where the Lagrange multipliers p, f3 and v are fixed respectively by the constraints of Eqs 21 [3] and [5] 



B. Maximization of the relative dynamical entropy with respect to some equilibrium reference process 



Another formulation that turns out to be more appropriate in many cases consists in considering the relative 
dynamical entropy with respect to the corresponding known equilibrium distribution P eq (£l[o,t]) of trajectories in the 
absence of the current 



= - E ln ^77^ ( 10 ) 



«[0,t] 
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The idea is then to maximize this relative entropy in the presence of the constraints concerning the normalization of 
Eq. 2] and the imposed current (Eq. [5]) to obtain 

V{n m ) = ^(Q^e-^+^KM]) (11) 

instead of Eq. [5] 

The notion of relative entropy with respect to some reference, also called the Kullback-Leibler entropy, is well- 
known in many areas. At a naive level, the idea is that the probability distribution used as reference constitutes some 
'prior' for the problem at hand : in the absence of any other constraint, the relative entropy is maximal for the 'prior' 
distribution. At a more fundamental level, the importance of the notion of relative entropy comes from its essential 
role in the Sanov's theorem concerning the theory of large deviations when formulated at the so-called 'level-2' (see 
for instance the review [27|). 

The interest to consider relative entropies in various areas of statistical physics has been discussed recently in (26[. 
In the present context of non-equilibrium steady states, the idea to take the equilibrium dynamics as the reference 
process is the basis of the Evans approach |LlT[l4| . 



C. Consequence : fluctuation relation concerning the integrated current 

It is clearly impossible to summarize here all the recent developments concerning the various 'fluctuation relations' 
that have been established in the field of non-equilibrium dynamics, and we refer the interested reader to some recent 
reviews (HHIII and references therein. Here we simply mention the minimum for our present purpose. 

The probability distribution of the integrated current is expected to satisfy some large deviation principle: the 
probability to have a given time-averaged value J{fl[o .*])/£ — j behaves at large t as 

Probfi^ml =j ] K e tG U ) (12) 

where G(j) < is called the large-deviation function. The typical value jt yp corresponding to the point where it 
vanishes G(j typ ) — is fixed here to Jo by the constraint of Eq. [M] Equivalently, the generating function of the 
integrated current J(f2[Q j t]) behaves for large time t as 

/ e AJ(n [0 , t] )\ K / $ e tW+G<j)) K e ^W (13) 

t— >-\-OG J t—^OQ 

where //(A) is the Legendre transform of G(j) 

/i(A)=max[Ai + G(j)] (14) 

3 

To characterize the irreversibility of the dynamics, it is interesting to compare the probabilities of a given trajectory 
f2[o,t] an d of its associate trajectory f2[o,t] obtained by time-reversal. In our present case, these two trajectories have 
the same energy E(£l[ Q ^) = .E(fi[ 0jt ]) but have opposite currents J(f)[ 0t ]) = — J(£l[ t ]). So Eq. [2] yields that the ratio 
of the probabilities between a trajectory and its time-reversed trajectory reads 

gffW = e 2»j(n [0tt] ) (15) 

More generally, any equilibrium dynamics has the property that two reversed-time trajectories occur with the same 
probability V eq (Q[ .t]) — T ?eq {^\o,t])- As a consequence, Eq. [TT]also leads to Eq. [T5] 

A direct consequence of Eq. [TS] is that the generating function of the integrated current defined as 

( e Mn [M )) = e XJ(n ^P(n [0yt] ) (16) 

f2[o,t] 

will satisfy the following symmetry (using the antisymmetry J(J7[ t ]) = — J(J7[ 0]t i) and the one-to-one change of 

id f2[ ,{]) 



variables between £l[o t t] and ^[o,t] 
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that can be rewritten for the function /u(A) introduced in Eq. [13] 

H(X) = fi(-X - 2v) (18) 

or equivalently after the Legendre transform of Eq. [14] 

G(j) = G(-j) + 2vj (19) 

In summary, the generalized Gibbs distribution of dynamical trajectories of Eq[9]or Eq [IT] directly leads to some 
fluctuation relation for the integrated current, as already discussed by Baule and Evans [12]. Independently of the 
context of the maximization of the dynamical entropy, the fact that fluctuation relations have actually for origin some 
generalized Gibbs distribution of space-time trajectories has been proposed by Maes [35| . 



D. Entropy production 

The notion of entropy 'production' is of course subtle. Since it is expected to measure the 'irreversibility' of the 
dynamics, it is natural to relate it to the probabilities between a trajectory and its reversed-time trajectory already 
introduced in Eq. [15] : a possibility [35l |36| is thus to define the entropy production S ,prod (il[ t j) of a dynamical 
trajectory firo,*] as the logarithm of the ratio of Eq. [15] 

5^ rf (n [0)t] )=ln^2Ml (20 ) 

Such a definition, which is in agreement with the usual expressions of the averaged entropy production derived for 
Markov chains [33, [38| , has for advantage that the fluctuation relation for the entropy production 



Prob(S prod ) _ SP ro d 



Prob(-SP rod ) 
is then a direct consequence of Eq. [5D] because 



(21) 



Prob(S prod ) = ^(^[o,t])S I S prod - In V ^ ^ j = P(ti{o,t])e SPr ° d 5 I S prod + In 



= e SPr ° d Prob(-S prod ) (22) 
For our present case, the definition of Eq. [20] leads with Eq. [15] to 



s prod (n m ) = in^£Ml ^ 2uj(n m ) 

P("[0,t]J 

i.e. it is simply proportional to the time-extensive trajectory current J(O[ t ]) that breaks the time asymmetry. 



(23) 



E. Discussion 



In this section, we have described the general idea of maximization of the dynamical entropy with constraints that 
leads to generalized Gibbs distributions for dynamical trajectories, and to some fluctuation relation for the integrated 
current. However, the derivation presented above remains a bit formal, since the space of dynamical trajectories has 
not been precisely defined. In particular, the important notion of 'stationarity' of the dynamics has been imposed 
only implicitly by the time-extensive constraints of Eqs [5] and [6] but no direct condition of stationarity has been 
explicitly imposed on 7 , (f2r 0) t]) (and we do not see how one could impose such a stationarity condition at this formal 
level). In the following sections, we thus consider well defined stochastic stationary dynamics generated by discrete- 
time Markov Chains or by continuous-time Master Equations to study the consequences of the maximisation of the 
dynamical entropy. 
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III. DYNAMICAL OBSERVABLES FOR A STATIONARY MARKOV CHAIN 



Discrete-time Markov Chains constitute the simplest formulation of stochastic dynamics presenting a dynamical 
entropy (Eq. [2]) that is extensive in time (Eq. [3]). In this section, we introduce the notations that will be useful in 
the remainder of the paper. 



A. Discrete-time Markov Chain 



Let us consider a system where possibles microscopic configurations are indexed by i, and where the probability 
Pt{i) to be in configuration i at time t evolves according to the discrete-time Markov Chain 

p t +i(j) = J2 Pt ^ Wi ^ ( 24 ) 

i 

where Wi^j represents the transition probability from i to j during the unit time interval, with the following normal- 
ization for each i 

E^ = 1 ( 25 ) 



3 

jst i 



The stationary state P (i) associated with this Markov chain satisfies 

P s \i) = Y,P st ij)W^ (26) 

3 

and the normalization 

E pst « = 1 ( 27 ) 



B. Probability of a dynamical trajectory in the stationary regime 



Suppose one starts at time t = in the stationary state P st of Eq. [25] The probability of a trajectory firo.t] — 
{io, H, it} to be in state io at time t = 0, in state i\ at time t = 1, in state %2 at time t = 2, ... , and in state it at 
time t reads 

V(n m = {i ,i 1 ,...,i t }) = P s \i )W iQ ^ il W il ^ i2 ..W it _ 1 ^ it (28) 
The normalization of Eq. [T] 

E E E •■• E v ( Q m = {*o, k: i*» = E ps *( io ) E wio-* E E w u-^ = 1 (29) 

i ii ii it io ii ii it 

can be easily checked using Eqs [51] and |2"T1 



C. Dynamical entropy associated with the distribution of trajectories 



Using the normalization and stationarity properties of Eq. [25] and Eq. [211 the dynamical entropy of Eq. [5] reads 
for the distribution V(£l\o,i\) of Eq [28l 

S dyn (t) = - E P(fi [ o, t] )ln7'(% t] ) 

f2[o,t) 

^EE-E^W^"^^ [ln(P* t (i )Wi -H 1 ...Wi t _ il _ >it _ 1 ) +lnWt_ 1 _ >it ] 



l 21 It 



= -EE-E pst («o^ I -w i 



H-2— Ht-i 



10 U U-l 



In (P^W^.-.WV^.J + E Wit-i-* lnW it -x- 



- 1) - E pst ^) E ^-1-* ln w^-* 



(30) 
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By recursion one obtains the following extensive behavior in the time t 

s d y n (t) = E p st (i) E in wuj (si) 

so that the entropy rate of Eq. [3] is given by the well-known result (see for instance the books [1, IHJ) 

Qdynf±\ 

h KS = lim -^ = -yP st (i)Yw i ^ j lnW i ^ j (32) 

» J 

The physical meaning of this formula is clear : it is the average, over the possible configurations i distributed with the 
stationary distribution P st {i), of the Shannon entropy associated with the transition probabilities Wi^j normalized 
with Eq. [251 



D. Energy associated with trajectories 

In statistical physics, one usually wishes to fix some constraint concerning the energy. For instance at equilibrium, 
the Boltzmann-Gibbs distribution can be obtained from the optimization of the static entropy in the presence of a 
constraint fixing the average energy (see Appendix 1X1 for a short reminder). In our present dynamical context, it is 
natural to associate to each trajectory f2[o,t] the integrated energy over the time interval 

E (tt[o,t] = {io,h, -,it}) = Eio + E h + ...E it _ 1 (33) 
so that its averaged value over the trajectories fWt] (using Eqs [231 and 1231) 

E dyn (t) = J2 H^t])E(n m ) = Y,Y,-J2 pSt ^ W ^n-W H _ 1 ^ t [E i0 +E h + ...£ it _ 1 ] 

f2[o,t] io ii it 

= Y / P St (^o)E lo + J2P St (n)E n + ... + Y / P St (^i)E H _ 1 =tJ2P st (i)Ei (34) 

io ii it-i i 

is extensive in time, and the ratio E dyn (t)/t corresponds to the static averaged energy computed from the stationary 
distribution as it should. 



E. Flux associated with trajectories 

In the field of non-equilibrium steady states, one is interested in particular in situations where a flux or current of 
some observable is imposed on the system. It can be a flux of particles, a flux of energy, or a flux of something else 
depending on the system under study and on the environment. 

In our present framework, we will consider that each transition i — > j is characterized by some contribution Ji^j 
to this current, with the antisymmetry property 

Ji—tj — (35) 

The integrated current J(fir 0i t]) associated with a trajectory f2[o,t] simply reads 

J(fo[0,t] = {*0, ilj •••) it}) = Jio^h + Jii-W a + ••• + Jit-i-Ht (36) 

so that its averaged value over the trajectories fifo t] (using Eqs [231 and [231) 

J dyn (t) = E P(^t])J^[o,t]) EE-E 7 '" 1 '^ 11 ' — ll " ■ - [Jio^n+Ji^i 2 +- + Jn-^u] 

= E pst ^ E wio-Mi + - + E pst ^-i) E w it _^ it j it _^ it 

io h h-i H 

=*E pst wE^ J ^ ( 37 ) 

« 3 

is again extensive in time as expected. 
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IV. RECOVERING EQUILIBRIUM FROM THE MAXIMIZATION OF THE DYNAMICAL ENTROPY 

The minimal requirement for any approach concerning non-equilibrium steady-states is of course to be compatible 
with the theory of the equilibrium. In this section, we thus describe how the maximization of the dynamical entropy 
of Eq. [2] allows to recover the equilibrium. As a comparison, we recall in Appendix [X] how the equilibrium is usually 
obtained via the maximization of the static entropy. 

A. Lagrange functional taking into account the constraints 

We consider all possible stationary distributions P st (i) satisfying the normalization 

M = Y / pst ( l ) = 1 (38) 

i 

and all Markov chains defined by some transition probabilities Wi^j , that satisfy the normalization condition of Eq. 
EHforalH 

M = E WUi = 1 (39) 

3 

and the stationarity condition of Eq. [55] for all i 

E< = P st (j)W^ - P st (i) = (40) 

3 

We also wish to fix the time averaged energy E dyn (t)/t (see Eq. IA5|) to some value E 

I ?^± = J2 pSt (i) E i = E G ( 41 ) 

i 

To optimize the dynamical entropy S dyri {t) of Eq. [31] in the presence of all these constraints, we introduce Lagrange 
multipliers p, ft, Xi and /Xj and we consider the functional 



l) _ p (E^)__ E \ _^ A . (A/ ,_ 1) _^ M . (E . 



: - E pst (*) E ln p (e pst « ^ - f 3 (e E > pst w - E ^j 

E Ai lE^- 1 ) -E^ (E pst (^-- pst w ) ( 42 ) 



B. Solving the optimization equations 

We wish to optimize the functional of Eq. 05] both over the stationary probabilities P st (i) 

= ^r-y = - E W *-H ln ~ P ~ ^ + W - E ^ ^ (43) 

3 3 



and over the transition probabilities Wi 



*->J 



(5* 

= ■ W — = P st (i) [- In - 1 - Mj] - A, (44) 

The Lagrange multipliers have then to be determined by imposing the various constraints. 
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Equation [44] yields 



Wi^j = e 1 H ^% (45) 



The normalization constraint of Eq. [39] for all i 

l = ^^=e^^e-"' (46) 



3 j 

imply that the Lagrange multipliers are given by 



\ i = P st (i)ln\^e-^- 1 ] (47) 
so that the rates of Eq. 25] can be rewritten as 



-I' 3 



The stationarity constraint of Eq. 001 yields that for all j 
so that the optimum found in Eq. 25] simply corresponds to [1, [n| 



k e_Mfc 



(49) 



W^ 3 =P st {j) (50) 

Eq. [43] becomes using Eq l49l and Eq [38l 

= -p-PEi - In P st (i) (51) 
so that using the normalization constraint of Eq. [38] one recovers the Boltzmann-Gibbs distribution 

P*U)= *~^L =P eq U) (52) 
where (3 is fixed by the constraint of Eq. |4T1 

C. Consequence for the probabilities of dynamical trajectories 

With the optimal solution obtained in Eqs [50] and [52] the probability of a trajectory f2[o,t] of Eq. [28] takes the 
simple form 

V*(Sl m ) = {io,h,...,it}) = P st (io)W io ^ il W h ^ i2 ...W it _ 1 ^ it = T^T^TtTT ( 53 ) 

where 

Z(P)=J2e-^ (54) 

i 

is the usual static partition function. So within the present approach, the probability distribution over dynamical 
trajectories is simply the Boltzmann-Gibbs distribution with respect to the trajectory energy i£(f2[o,t+i]) of Eq. IMl 

-/9E(n [0it+1] ) 

V ec! (fl l0 t] ) = ^— (55) 
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The appearance of E^fi^t+ij) (instead of E(Cl\Q it ]) found in Eq. [9] via the formal argument ) is a consequence of the 
choice of Eq. [Ml for the energy for the discrete Markov chain : the energy E(i t ) of the last state i t will contribute to 
the trajectory energy only during the next time interval [t,t + 1], but it nevertheless appear in the probability of the 
trajectory fi[o,t] ending in state i t . This slight problem thus comes from the discrete nature of the dynamics, and is 
not expected to play an important role : it is only a finite boundary term that will become subleading in the large 
time limit t — > +oo with respect to the time-extensive energy of the whole trajectory. 

An essential property of this distribution of trajectories is the symmetry by time-reversal : the probabilities of 
a given trajectory f2[o,t] = {io, H> •••> it-i^t} arid of its associate trajectory O[o,t] = {it, H-i, *o} obtained by 
time-reversal are equal 

P( n [o,t] = {io,h,-,it}) = 1 ^ 
^(^[o,i] = {it,it-i, —,ii,io}) 

D. Discussion 

In this section, we have described how the maximization of the dynamical entropy allows to recover that the 
stationary distribution follows the Boltzmann-Gibbs distribution (Eq. [5^]) . However the result of Eq. 1501 concerning 
the transition probabilities Wi^j may be surprising at first sight : this solution means that the new configuration j 
is chosen with the probability P st (j) and is completely independent of the initial state i. Physically this corresponds 
to a very coarse-grained dynamics with no memory. It is thus clear that Eq. [50] does not represent an effective 
'microscopic' dynamics, but represents an effective dynamics on some macroscopic time r. The corresponding entropy 
rate per unit time of Eq. [3] 

\ i / j j 

then exactly coincides with the static entropy of Eq. IA11 which seems natural if one wishes this approach to be 
equivalent to the maximization of the static entropy that one uses for the equilibrium (see Appendix IA"|) . This 
discussion suggests the following interpretation of this type of computation. 

E. Final formulation of the physical meaning of this approach 

The statistical physics theory of equilibrium is usually based on some 'ergodic' hypothesis, stating that time- 
averages of observables A(i) will converge in the infinite-time limit towards averages computed with respect to the 
Boltzmann-Gibbs distribution P eq (i) of Eq. [521 

lim - f dtA(i(t)) = E A{i)P^{i) (58) 

Since the right-handside contains only the Boltzmann-Gibbs distribution P eq (i) — er^ Ei /Z(j3) and no other infor- 
mation about the dynamics except the conserved energy, this means that the precise form of the 'true' microscopic 
dynamics becomes completely irrelevant at large time. Loosely speaking, this means that there exists some finite 
macroscopic correlation time T corre i, beyond which the dynamical correlations have been lost in practice, so that the 
large-time interval t can be decomposed into t/T CO rrei quasi-independent time intervals, and Eq. [58] becomes possible. 
On the contrary, if the correlation time T corre i of the 'true' microscopic dynamics is infinite, Eq. 1581 cannot really be 
satisfied since other dynamical informations besides the conservation of energy remain relevant forever. 

This discussion suggests that the analysis presented above based on the maximization of the dynamical entropy, 
with the result of Eq. 1501 actually describes what happens at this coarse-grained macroscopic scale T corre i, i.e. the 
unit-time of the Markov Chain of Eq. [MJ should be interpreted as r corre ;. Then we have shown above that one recovers 
the Boltzmann-Gibbs distribution, and thus all the 'static' properties that can be derived from it. 

However in the statistical physics of equilibrium, once one has understood the properties of the Boltzmann-Gibbs 
measure over configurations, one can become interested into the 'equilibrium dynamics'. However, one does not wish 
to return to the 'true' deterministic microscopic dynamics which is usually very complicated for systems with a very 
large number (like 10 23 ) degrees of freedom, because all the details of the dynamics have proven to be irrelevant at 
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large time scale. So one introduces an effective stochastic microscopic dynamics that is compatible with the known 
properties on large time scales. For definiteness, let us consider a microscopic Markov Chain 

Pt+At(j) = ^2pt(i)wi^j (59) 

i 

where Wi^j represents the transition probability from i to j during the microscopic time interval At, so that Wi^j are 
now non-zero only if the configurations i and j are sufficiently close in configuration space. For instance in a system of 
N spins with 2 N configurations, one may require that is non-zero only if j can be obtained from i by the flip a 

single spin. More generally, this notion of elementary moves has to be defined in an appropriate way for each type of 
models. Then the question is : beyond this locality requirement, how should these microscopic transition probabilities 
Wi^j be chosen to be compatible with the properties known on large time scales ? The first obvious requirement is 
that the microscopic Markov Chain of Eq. [59l should have for stationary state p st (i) the Boltzmann-Gibbs distribution 
which is known to be the stationary state on macroscopic times 

p st (i) =P £ i(i) (60) 

However this is not the only constraint, since one also wishes to reproduce the essential time-reversibility property of 
dynamical trajectories of Eq. 1561 For this, it is sufficient to impose that the probability of the elementary microscopic 
trajectory w A t = {«o, h} 

Prob(uj At = {i , ii}) = p^w^^ (61) 

is equal to the probability of the reversed trajectory wa ( = {i\, , io} 

l = Prob(uj A t = {io,ii}) = pfo)^o->n 
Prob(uiAt = {h,,io}) p{h)wh^i 

Taking into account Eq. [Ml this leads to the well-known detailed-balance condition 



<U _ P(h) _ P eq (n) 
<% i0 P(io) P eq (io) 



»o-»i _ W = j W _ e -/3(B(ii)--B(io)) C531 



Besides this detailed-balance constraint, there is still some freedom to choose the effective microscopic transition 
probabilities Wi_>j, but one usually considers that they are equally valid, in the sense that they will all reproduce 
the essential large-time properties of convergence towards the Boltzmann-Gibbs distribution while preserving the 
time-reversibility of dynamical trajectories. 



F. Is it possible to add locality constraints within the maximization procedure ? 

As explained above, the requirements on effective local Markov chain dynamics of Eq. [55] are derived from the 
macroscopic equilibrium properties. A natural question here is whether these requirements can be instead directly 
derived from a maximisation of the dynamical entropy in the presence of locality constraints on the . We have 
now the following picture : configurations are the nodes of a network, and the links are present between configurations 
that are related by an elementary local move. As a first example, in a spin models with N spins and 2 N configurations, 
one may consider a single spin-flip dynamics, where each configuration has N neighbors (corresponding to the flip of 
one of its N spins). As a second example, in lattice gases models with hard-core interactions, the number of neighbors 
of a configuration will be given by the number of possible local moves from this configuration, and will thus depend 
on the configuration. 

Whenever the connectivities (i.e. the numbers of neighbors) are configuration-dependent, it is clear that one cannot 
recover the equilibrium from the maximization of the dynamical entropy in the presence of locality constraints, as 
can be easily understood on the simple infinite temperature case (5 = : 

(i) At infinite temperature, the problem of the maximization of the dynamical entropy is completely equivalent to 
the Maximal Entropy Random Walk (MERW) on arbitrary networks studied in details in (T^], with the following 
main conclusion : in networks where the connectivities of the various nodes are not all the same, the MERW tends 
to visit more the sites with higher connectivities than the sites with lower connectivities, because sites with higher 
connectivities are associated with a bigger number of choices for the next step of the random walk. 

(ii) On the other hand, at infinite temperature case (3 = 0, the Boltzmann-Gibbs distribution simply corresponds 
to the uniform distribution over all configurations, independently of their connectivities from the point of view of the 
local dynamics. 
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Our conclusion is thus that the choice of the connectivities that one wishes to impose on the local dynamics has a 
too strong effect on the dynamical entropy that one wishes to maximize, whereas the maximization of the dynamical 
entropy at a macroscopic time scale allows to recover the equilibrium, as explained above in section IIVEI In the 
non-equilibrium case, this discussion on local dynamics will be the subject of sections I VII and IVII1 but in the next 
section, we first describe what happens in the absence of locality constraints. 

V. NON-EQUILIBRIUM STEADY STATE WITH AN IMPOSED CURRENT 
A. Lagrange functional taking into account the constraints 

With respect to the previous section I IV A[ we now add another constraint concerning the flux of Eq. l37l 

Jdyn^ 



J2 p st (*) E w ^ = J ° ( 64 ) 



t 

* 3 

So we introduce this constraint with a new Lagrange multiplier v into the functional of Eq. [42] to obtain 

i i ^ 

= - E pst ^ E ln ^ - p(E pst ^ - !) - p(E EiPSt ® - E o) 

i j i i 

- E A * (e w ^ 1 j - E * (e pst ^ w ^ - pst w j + " (e prt w E w^j^ Jo ] (65) 

B. Solving the optimization equations 
Equations [43] and [44] are modified into 

o = =-J2 ln w t-n -p-0Ei + m - »kWi^ k + f E w <-n J <-n ( 66 ) 

j k j 

and 

= -=^— = P st (i) [- In - 1 - m + tyJi^j] - \ (67) 

Equation [67] yields 

Wir+j = e - 1 "^ +!/j4 ^-p^) (68) 
The normalization constraint of Eq. [JH] for all i 

1 =JlVi^ = e^^)J2 e ^ 3+VJi ^ ( 69 ) 

leads to the Lagrange multipliers 



Aj = P s *(i) ln [ J2 e-^~ x+vJi ^ ] (70) 

so that the rates of Eq. 0S] become 



3 



Wi^3 = ^—^77-7 (71) 
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To analyze the stationarity constraint of Eq. [40] for all j 



it is convenient to introduce the notations 



k c 



Zi 



Vj 

so we may now eliminate all fij in terms of the yj . In particular 



(72) 



fc 

and 

% - E ^e^-- (74) 

to rewrite Eq. [72] as 



(75) 



^ = £^f^ (^) 



is somewhat the 'dual' of Eq. [75] The transition probability of Eq. [7T] now reads 

= K dl e vJi ^ (77) 

ZiVj 

and Eq. 1661 becomes 

= -p-pE i - In P st (i) + Info) + £ W^- [- In - p j + vj^] 

j 

= -p- pEi - \nP st {i) + Info) + ln(zj) (78) 

leading to 

P st (i) = yiZie-r-M (79) 

Plugging Eq. 1751 into Eqs [74] and [76] leads to 

I/i = Swe-'-^e"*^ (80) 

i 

and 

z i = Y l ^e- p - pBh e vJ ^ (81) 

He 

C. Optimal stationary distribution P st (i) and transition probabilities Wi-+j 

In summary, we have obtained that the optimal stationary distribution P st (i) and transition probabilities 
follow the form 

P st (i) =y i z i e-"-^ 

W = ?L e -P-PB}+»Ji^i (82) 
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where the yi and are positive variables satisfying respectively the equations 

v 3 = Y,y* e ~ p ~ pEteVj ^ 3 ( 83 ) 

i 

that correspond to the stationarity constraints P st (j) — ^2iP st {i)Wi^j and the equations 

Zl = Y, eUj ^ 3e ~ p ~ 0E ° z 3 ( 84 ) 

3 

that correspond to the normalizations 1 = ■ Wi->j , and 

i = Y,yi^"^ Bi (85) 

i 

that correspond to the normalization 1 = J^. P st (i). To see more clearly what this structure means, it is convenient 
to introduce the bra and ket notations and to set 

ee e^ 1 < > (86) 
Then in terms of the matrix M defined by the positive matrix elements 

< i\M\j >= e -^e vJ ^ie-^ E i (87) 

Eqs 1551 and 1541 mean that < L\ and \R > are respectively positive left-eigenvector and right-eigenvector of the non- 
symmetric matrix M 

e p < L\ =< L\M\ 

e p \R > = M\R > (88) 
associated with the highest eigenvalue e p of the matrix M (Perron- Frobenius), with the normalization given by Eql85l 

e p =<L\R> (89) 



The equilibrium case discussed in the previous section ITVl corresponds to v = 0, Hi = 1 = Zi, < L\i >=< i\R >= 
e~2 E i. In the following to discuss what happens for v > 0, we will use the notations yi and Zi that appear in Eqs [521 

D. Consequence for the probability distribution of dynamical trajectories 

From the optimal solution of Eq|52J one obtains that the probability of a trajectory fi[o,t] of Eq. E51 takes the form 
V(n m ) = {*o,*i,..., it}) = P st (io)W io ^ il W il ^ i2 ...W it _ 1 ^ it 

"ia z 'h "it— i 

= y loZit e-K E ^+ E ^+---+ E H)+^{j^i+-h 1 ^^+---+.J H -i^n) e ~{t+^P 

= y ^0 z ^t e-' 3B ( ^2 l«• t + 1 ^) +, ' J ( n [ . t ^)e-( t+1 )' , (90) 

So it is not exactly as simple as Eq. |H] : besides the expected Boltzmann-Gibbs factors involving the trajectory energy 
-E(fWt+ii) (already found for the equilibrium case in Eq. I55p and the trajectory current J(fl< 0>t i), and besides the 
normalization insured by the choice of p in the last factor, there remains the non-trivial boundary prefactor yi Zi t 
: we believe that this term comes from the stationary constraint on the dynamics that we could impose within the 
Markov chain framework, whereas we were not able to impose this stationary constraint at the formal level discussed 
in section [TXJ However, this supplementary prefactor is a 'boundary term' containing only the initial and the final 
configurations. As a consequence, it is not expected to grow in time, and in the large time limit t — > +oo, it will 
become subleading with respect to the time-extensive terms present in the exponential, since the trajectory energy 
and the trajectory current are extensive in time by the imposed constraints. (Note however that in some cases with 
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unbounded phase space, the 'boundary terms' may diverge and remain important even in the large-time limit, see 
section 5.3 of the review [29[ and references therein for more details). 

Our conclusion is thus that the formal solution of Eq. [9] neglects only the boundary terms of Eq. [90] and thus 
captures correctly the dominant terms in the large-time limit t — > +oo. In particular, it contains the factor that is 
responsible for the fluctuation relation of the integrated current discussed in section III CI and for the dominant term 
of the entropy production discussed in section III PL We will obtain the same conclusion in the presence of locality 
constraints on the dynamics that we consider in the next section. 

VI. EFFECTIVE STOCHASTIC MICROSCOPIC DYNAMICS IN THE NON-EQUILIBRIUM CASE 

As explained in section IIVFI there exists some difficulty to recover the equilibrium if one tries to maximize the 
dynamical entropy associated with a Markov Chain containing locality constraints, as a consequence of the possible 
configuration-dependent connectivities that one wishes to impose on the possible elementary moves. However, as 
recalled in section IIVEI the requirements on effective stochastic microscopic dynamics (Eq. I59|) to describe the 
equilibrium dynamics are well understood, and lead to the detailed balance condition of Eq. 1631 In this section, 
we will thus follow the point of view of the Evans approach [lTl - flij : we consider that a local equilibrium dynamics 
generated by some Markov Chain satisfying detailed balance is known, and we wish to determine the non- 

equilibrium appropriate modified Markov Chain u>j_>j in the presence of an imposed current. Instead of maximizing 
the 'full' dynamical entropy of Eq. [5] we will thus maximize the relative dynamical entropy with respect to the 
equilibrium dynamics, that we have introduced in Eq. 1101 



the relative dynamical entropy of Eq. [TO] with respect to the equilibrium dynamical trajectory corresponding to 
has for time-extensive behavior 



A. 



Maximization of the relative dynamical entropy with respect to the equilibrium 



For the microscopic Markov Chain 




(91) 



idyn 
'rel 



(*) = -E ^[o,*]) ln 



v e i(n M ) *->+«, At 




(92) 



The maximization of this relative entropy in the presence of the constraints concerning the normalizations 




(93) 




the stationarity condition 




(94) 



3 



and the current 




(95) 




(96) 
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its largest eigenvalue e p , and the corresponding right and left positive eigenvectors \R > and < L\ 

M\R> =e p \R> 

< L\M =e p <L\ (97) 

normalized with 

< L\R >= e p (98) 

With these notations, the optimal solution reads 

p st (i) = e ~P <i\R>< L\i> 

In the equilibrium case v = 0, the matrix M reduces to the generator of the equilibrium Markov Chain My = w^j, 
so the maximal eigenvalue corresponds to e Pc<! = 1, the right and the left eigenvectors are simply < i\R eq >= 1 and 
< L eq \i >= P e «(i). 



B. Consequences for the probability distribution of dynamical trajectories 



From the solution of Eq. [Ml one obtains that the probability of a trajectory lu = {i , i\, i n }) takes the form 
V(ui = {io,ii, —,i n }) = p st (i )w la ^ il w il ^ l2 ...w ln _ 1 ^ ln 



< i \R X L\i > w^ h e 



-p+vJi, 



< h\R > 



< L\i >< i n \R > 



P eq (io) 



<i \R> 

v e «{u = {i Q M,-MV J{u) z~ [n+1)p 



< in\R > 

< i n -i\R > 
(100) 



Again, it is not exactly as simple as Eq. 111! besides the equilibrium trajectory probability, the expected Boltzmann- 
Gibbs factors involving the trajectory current J(Qro,t]), and besides the normalization insured by the choice of p in 
the last factor, there remains a boundary prefactor that depends only on the initial and final configurations. But this 
boundary factor is expected to become subleading in the large time limit t — > +oo, as already discussed in section 

EE 



C. Link with the Bayesian approach of Evans 



In [ll(, Evans has introduced an approach called 'Non-equilibrium Counterpart to detailed balance'. The general 
idea is clearly the same as in the present article, namely the maximization of the dynamical entropy in the presence 
of the appropriate constraints. However, the way of reasoning is slightly different and thus gives other useful points 
of view : Bayes theorem is used to analyse the properties of an elementary trajectory belonging to a macroscopic 
trajectory satisfying the flux constraint with respect to the equilibrium dynamics. The outcome is that the transitions 
w dnven m d r i ven case should have the following form (see Eq. (24) of [llj]) 

w driven _ ^er^ g! , J i ^ j At+ qj (u)- qi {v)-Q(v) At (101) 

From the comparison with Eq. |MJ one obtains the following correspondence : Q{v) At corresponds to the normalization 
factor p, whereas qj(v) corresponds to In < j\R >. The physical interpretation proposed by Evans is that the factor 
qj(v) measures the 'willingness' of state j to accept a future flux. We refer the reader to the very interesting series of 
articles [TTrll^ l to have more detailed explanations and to see various examples of application. 



VII. CASE OF CONTINUOUS-TIME MASTER EQUATION 



Up to now we have considered the case of discrete-time Markov Chains, that constitute the simplest framework 
to define probabilities of dynamical trajectories and their Shannon entropy of Eq. O However, many studies on 
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non-equilibrium systems prefer to consider stochastic dynamics that are generated by some continuous-time Master 
Equation 



dtPt(j) = ^2p t {i)ki^j -pt(j) 



Mi 



(102) 



In this section, we describe how the results of the previous sections should be adapted for this case. 



A. Dynamical entropy for the continuous-time master equation 



Let us first recall how the master Equation (j 102[) can be obtained as the continuous limit At — > of the Markov 
chain of Eq. [9TJ In the limit where the elementary time step becomes small At —> 0, it is natural to assume that the 
transition probability from i to another state j ^ i becomes proportional to At 



ki—tiAt 



where fc<_>j represents the transition rate per unit-time from i to j. The normalization of Eq. 
transition probability to remain on site i takes the form 



At^O 



1 - Atk out (i) 



where 



(103) 

| yields that the 
(104) 

(105) 



represents the total exit rate out of state i. In this limit At — > 0, the finite-time Markov chain of Eq. [!|1] thus 
becomes the master Equation 1 1021 

However as explained in [17], the continuous time description leads to some difficulties when one considers the 
dynamical entropy of Eq. [2j Indeed, the naive continuous limit of Eq. [32] becomes 



Sdyn{t) 



At- 



* 3 

o At ^ y ' 



i— >3 m w i— >j 



J^ki^jAtlnik^jAt) + (1 - Atk out (i))\n(l - Atk out {i)) 



\t-xt ^— ' 



"-out I V 



(106) 



i.e. the elementary time At remains in the argument of the logarithm to have a non-dimensional quantity. This 
difficulty to define the 'absolute' entropy in problems involving continuous variables can be usually circumvented by 
considering the 'relative' entropy with respect to some reference. In our present framework, the reference will be the 
corresponding equilibrium master equation defined by transition rates satisfying the detailed balance condition 
(EqESD 



(107) 
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The continuous-time limit At — )• of the relative dynamical entropy of Eq. [92] reads 



s*T(t) 



At^O 



+ (1- Atfc out (i))ln 



A/- 



xi ^-^ in ( p?" 1 ) + ~ X! 



1 - Atk ou t(i) 



+ fc «t(«) - fe^t(0 



z, e <? 



(108) 



So this relative dynamical entropy is well-defined for continuous time Master Equations, and can be used in maxi- 
mization procedures. 



B. Maximization of the relative entropy with the appropriate constraints 



We wish to maximize the relative dynamical entropy of Eq. 11081 with respect to all stationary states p st (i) satisfying 
the normalization (as in Eq. I38|) 



jv=Xy(i) = i 



(109) 



and with respect to all possible transition rates fcj->j with j ^ i (note that here there is no normalization equation 
as Eq[39] since it has already been taken into account in Eqs 11041 and !105p that have p st (i) as stationary distribution 
(equivalent of Eq. [40]) 



in the presence of the following flux constraint (equivalent of Eq. I95|) 

= X^ W E br+i^i = ^ 







So we introduce the following functional with appropriate Lagrange multipliers 



* (t) - P (N 1) - E W (^) + - f J o 



t 



E ln ( tw 1 ) + E fc *^ - E 



-pcE^w- 1 ) 



EMi(Ep si (i)^-p st (i)E^ 



The optimization with respect to with i j 



Sty 



P st (i) 



->3 



In' ^ 



E pSt w E k ^j J i^j - J 



yields 



(110) 



(111) 



(112) 



(113) 



(114) 
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which is the appropriate continuous limit of the Markov chain result of Eq. [99] and of the Evans result cited in Eq. 

[MI 

The optimization with respect to p st (i) 
can be simplified using Eq. 11131 into 



ki—tj 



ln I lJq~ + 1 - Mi + Mi + "Ji-H 



J2 k ^ 3 -p ( 115 ) 



or equivalently in terms of the total exit rates introduced in Eq 11051 one obtains 

k 0Ut (i) = k e l t (i) + P (H7) 

Physically, this means that for all configurations i, the exit rate is shifted by the same amount p with respect to the 
equilibrium case. Eq. 11171 has been previously obtained by Baule and Evans in [l2j within their slightly different 
perspective, in order to obtain a set of invariant quantities for shear flows and to compute the non-equilibrium rates 
via some network rules (see for more details). 

In contrast to Eq. 11131 Eq. 11171 can appear as 'new' with respect to the discrete-time Markov chain result of Eq. 
[Ml Another apparent difference is that the discrete-time Markov chain result of Eq. [M] contains both the stationary 
distribution p st (i) and the transition probabilities that are determined together, whereas here the two types of 

equations (Eq. 11131 and Eq. I117|) concern only the transition rates, and p st {i) does not appear anymore. However, this 
absence of p at {i) is only apparent, because p st (i) is of course directly determined by the rates k^j via the stationarity 
condition of Eq. 11101 To make clearer the correspondence with the discrete-time Markov chain result of Eq. [Ml it is 
thus useful to reformulate the continuous-time master equation solution as follows. 

C. Reformulation of the solution in terms of an eigenvalue problem 

The aim of this section is to reformulate the solution found above in Eqs 11131 and 11171 in terms of an eigenvalue 
problem to see the similarity with the discrete-time Markov chain solution of Eq. [99] 

To replace the matrix M of Eq. 1961 it is convenient to introduce the matrix N defined by the matrix elements 

Nu =-tCi=-C(0 (H8) 

Setting =< j\R >, Eq. 11131 becomes 

k^=N v ^g (H9) 

whereas Eq. II 171 becomes 

p = K U S) KlS) = E ^Tijf^ + N « = 77^7 E < >< j\ R >= < <Tr> ( 12 °) 

j^i 3 

meaning that R > is a right eigenvector of the matrix N corresponding to the eigenvalue p 

N\R>=p\R> (121) 
Let us now rewrite the stationary equation of Eq. II 101 using Eq. 11181 Eq. 11191 and Eq. 11171 

= J^P^ki^ - P st (j)k out (j) =J2p st (i)N ij ^§^ ~P st (j) (Kltt) + P) (122) 



20 



It is thus convenient to set 

P st (i) =< i\R >< L\i > (123) 

to rewrite Eq. 11221 as 

= < L \ l > N v+ < L \3 > ( N Ji ~P)=J2 <L \ i >< W >~P< L \3 >=< L \ N \j >-P< L \j > ( 124 ) 

meaning that < L\ is a left eigenvector of the matrix TV corresponding to the eigenvalue p 

<L\N = p<L\ (125) 

In summary, the solution of Eqs 11131 and 11171 for the continuous-time master equation can be written in terms of 
an eigenvalue problem for the matrix N introduced in Eq. 11181 as follows : 



p st (i) =<i\R><L\i> 

<J\R> 

< i\R > 



k^j = i% * (126) 



where \R > and < L\ are the right and the left positive eigenvectors associated with the largest eigenvalue p of the 
matrix N, with the following normalization 

1=<L\R> (127) 

The correspondence with the discrete-time Markov chain result of Eq. [W] is now clear. As a final remark, let us 
mention that in the equilibrium case v = 0, the matrix N is the generator of the equilibrium dynamics, so the maximal 
eigenvalue corresponds to p eq — 0, the corresponding right and the left eigenvectors being simply < i\R eq >= 1 and 
< L eq \i >=p e "{i). 



D. Relation with the 'constrained dynamics' introduced to study large deviations 

The reformulation in terms of an eigenvalue problem described in the previous section is also useful to make the 
link with the the 'constrained dynamics' that have been introduced in various studies concerning large deviations of 
stochastic processes described by master equations (see for instance the recent works [39rl42| and references therein). 
In this context, the point of view is as follows : one considers some 'true' dynamics and one is interested into large 
deviations properties of some observable like the current. To compute the probability of rare events giving rise to an 
anomalous current for the 'true' dynamics, it is useful to introduce an 'auxiliary' dynamics that takes into account 
the conditioning to produce a given anomalous current (see [39UI2I ] for more details). The transition rules of this 
auxiliary dynamics and its stationary state are given in terms of left/right eigenvectors of a modified operator via the 
same formula written above, see for instance Eqs (2.15) (2.17) (2.18) (2.19) in [H or Eqs (2.16) (2.18) and (2.19) in 
[40l | : the exchange of the roles of the right and left eigenvectors with respect to the present notations comes from the 
different choice in the writing of matrices, since [3^ . |40| have chosen the 'quantum mechanical' convention where the 
initial state is on the right and the final state is on the left. The fact that exactly the same equations appear can be 
understood as follows. In the present paper, we have described how the maximization of the dynamical entropy yields 
Eq. [H] at a formal level, and give the results of the previous section when applied to an explicit master equation. 
But of course another possibility could be to take the formal solution of Eq. [TT]as a starting point, and to derive the 
consequences of this reweighting of trajectories for the specific case of a master equation : this is exactly the route 
followed in the large deviation studies mentioned above (39U421 ] and so one should indeed obtain the same results 
by consistency. However, even if the equations are exactly the same, the physical interpretation of this modified 
dynamics is different : in the large deviation studies |39h42| . this modified dynamics is usually presented only as 
a useful technical tool to better understand the large deviations of the 'true' dynamics, whereas in the approach 
summarized in the present paper, one considers that the modified dynamics is the 'real' dynamics in the presence of 
an imposed current. 

An interesting output of this comparison with large deviation studies is that it provides specific studies of this 
modified dynamics in models different from the examples considered in the works of Evans [11H14| : the auxiliary 
dynamics corresponding to the large deviations of the current of the Asymmetric Simple Exclusion Process has been 
studied in various situations or regimes in [39l IZll . |42| , whereas constrained dynamics for the Glauber Ising chain has 
been considered in 40]. Since the change of measure is a standard tool in the large deviation theory, many other 
examples can actually be found in the huge number of works concerning large deviations for stochastic processes. 
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VIII. CONCLUSIONS AND PERSPECTIVES 



In this paper, we have argued that if one wishes to formulate a general principle based on the maximization of some 
notion of 'entropy' for non-equilibrium steady states, the Shannon entropy associated to the probability distribution 
of dynamical trajectories is definitely the most natural, as first proposed by Filyokov and Karpov in 1967. The 
general idea is then to maximize dynamical entropy of Eq. [2] in the presence of appropriate constraints, including the 
macroscopic current of interest, via the method of Lagrange multipliers. We have tried to give a self-contained and 
unified presentation of this type of approach. We have first described at the formal level how this maximization leads 
to generalized Gibbs distribution for the probability distribution of dynamical trajectories, and to some fluctuation 
relation of the integrated current. We have then discussed in detail the cases of well defined stochastic dynamics 
generated either by discrete-time Markov Chains or by continuous-time Master Equations. In the cases where the 
use of the 'full' dynamical entropy of Eq. [2] leads to some difficulties, we have shown how the use of the 'relative' 
dynamical entropy of Eq. [TU] allows to solve the problems. The obtained results are in full agreement with the Evans 
approach called 'Non-equilibrium Counterpart to detailed balance' [ITl - flij ]. but give a slightly different perspective. In 
particular, we have obtained that the stationary distribution and the transitions probabilities or transitions rates could 
be obtained in various cases from an eigenvalue problem. Finally, we have explained the link with the constrained 
dynamics that are often introduced in large deviations studies of stochastic processes. 

In this article, we have remained at a very general level with a dynamics visiting a series of configurations, to see 
more clearly what general properties could emerge, since it is at this level of generality that the statistical physics 
theory of equilibrium is formulated. However it is clear that it would be interesting in the future to study more 
precisely how this type of approach can be applied in various models of interest, and to discuss whether the obtained 
dynamics can be considered as 'real'. An important issue is of course to compare with other approaches that solve 
exactly some non-equilibrium dynamics. In the field of quantum models, the non-equilibrium steady states have been 
found to be generalized Gibbs states, but they involve an infinite number of conserved quantities coming from the 
integrability of the considered models [43ll44j]. For non-integrable models, one may thus hope that the non-equilibrium 
steady states correspond to much simpler generalized Gibbs states. 

As a final remark, we should stress that the idea of maximizing some dynamical entropy to describe non-equilibrium 
steady states has been argued here to hold for 'physical systems' for which, in the absence of any imposed current, the 
notion of equilibrium exists and corresponds to the maximization of the usual 'static' entropy of statistical physics, 
and for which effective stochastic models satisfying detailed balance are well accepted to describe the equilibrium 
dynamics. But of course, in the field of non-equilibrium, many models of interest are inspired not by physics, but by 
biology (like predator-prey models), sociology (like road traffic models), politics (like voter models) etc... In all these 
cases, it is clear that one can choose arbitrarily the microscopic rates to model the considered phenomenon as one 
wishes. 
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At equilibrium, one is interested into the equilibrium probabilities P eq (i) of occupations of configurations i. The 
Shannon entropy associated with the equilibrium distribution is 
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Appendix A: Reminder on the maximization of the static entropy for the equilibrium state 




(Al) 



(i) If the only constraint is the normalization, 




(A2) 



22 



the maximization of the entropy of Eq. IA1I with the constraint of Eq. IA2I can be obtained by introducing a Lagrange 
multiplier A and the functional 

$ A = S eq — X(N — 1) = -^P eq {i)\nP eq (i) - \(^P eq {i) - 1) (A3) 

i i 

The optimization with respect to P eq (j) yields 

0=^ = -mP- (j )-l-A (A4) 

leading to P e q{j) = e~ 1_A , where A is determined by the normalization constraint of Eq. IA2I So P e q(j) is simply 
uniform over all states. 

(ii) If in addition to the normalization of Eq. IA21 one imposes also a constraint for the averaged energy 

<E>=Y,E l P eq (i) = E (A5) 

i 

one has to introduce another Lagrange multiplier /3 and the functional 

*A,/J = S eq - X(N - 1) - /?(< E > -E ) 

= -J2 P eq ^) ^P eq {i) - KJ2 ~ l ) ~ £(E E t P eq {i) - E ) (A6) 

i i i 

The optimization with respect to P e q(j) yields 

= 7FT7 = - ln - 1 - A - pEj (A7) 



leading to P eq {j) = e 1 A l3Ej , where A is determined by the normalization constraint of Eq. IA21 and /3 by Eq. IA5 
Then P eq (j) follows the Boltzmann-Gibbs distribution 



e -PEj 

This derivation of the Boltzmann-Gibbs distribution, introduced by Jaynes (23l |. has the advantage to be very simple 
and to really show what the principle of maximum entropy contains. For more details on the interest to consider 
statistical physics from the point of view of the Shannon information entropy, we refer the reader to the 'old' book 
[46j and to recent presentations [47j and references therein. 



Appendix B: Technical simplifications for an alternate Markov chain 

In their original paper 0] , Filyokov and Karpov have not considered the homogeneous Markov chain of Eq. [Ml but 
have focused instead onto an alternate Markov chain, that has been also reconsidered recently by Favretti [1(|. In 
this Appendix, we show how this alternate Markov chain framework yields technical simplifications with respect to 
the solution described in section fVl concerning the homogeneous Markov chain. 

1. Alternate Markov Chain 

In this section, we consider the alternate Markov chain, where the transition probabilities Wi^j takes alternatively 
two sets of values Ai^j and , so that the probability of a trajectory of Eq. [55] now becomes 

- {*o,*i,».,»2t}) = P even {io)A io ^ 1 B il ^...A i2t _ 2 ^ i2t _ 1 B i2t _ 1 ^ t (Bl) 

with the normalizations (as in Eq. I25[) 

K B =Y. B ^ = 1 ( B2 ) 
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The stationary states P even (i), P odd {i) at even and odd times associated with this alternate Markov chain satisfy 



with the normalizations 



Sf en = P° dd (j)Bj^i - P even (i) = 

3 

T,° dd = J2 P even {j)A^i - P odd (i) = 

3 

jyeven _ ^ ' peven^ _ ^ 
i 

jV odd = P° dd {i) = 1 



The dynamical entropy of Eq. [3T] becomes 



S dyn (2t) = -t 



P even (i) E ln + E E ln B 



The averaged energy is now fixed by the constraint 

E d v n (2t) _ 1 



2t 



P even (i)Ei + J2 P° dd (i)E l 



En 



that replaces Eq. [JTJ 



(B3) 



(B4) 



(B5) 



(B6) 



2. Simplification concerning the current constraint 

in sua, 

the interest was in an energy current flowing through the system : the advantage of this formulation 
with an alternate Markov chain is that one may consider that during the evolution described by A, there is a flow of 
energy entering the system, whereas during the evolution described by B, there is a flow of energy coming out of the 
system. To remain in a stationary state, these two flows have to be equal, and can be simply expressed in terms of 
the difference of averaged energies between the even and odd stationary states 

= J- (i) j2 A^m) E{i)) = ]r E(j)p° dd (j) - Yl mp even (i) 

i 3 3 i 

jout = po dd {i) Y B^Eii) E(j)) = Y E(i)P° dd (i) - E(j)P even U) = Je (B7) 

i 3 i 3 

In the following, we consider more generally the case of a current Jk that can be written similarly as the difference 
between some observable K between the even and odd stationary states (K is the 'charge' associated with the current 
Jk) 

jp = Y,p even {i)Y, A ^ K (j) - = Yl K (3)p odd U) -Y K ^ peven w 

i 3 3 i 

jout = j2P odd (i)j2 B ^(K(i) - Km =Y d K{i)p° dd {i) -J2 K U)p even (j) = j 



in 

K 



(B8) 



so that the constraint of Eq. [M] can be replaced by 

j d y n (2t) 



t 



J2 K tt) [P° dd (i) - P even (i)] 



(B9) 
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3. Optimization of the Lagrange functional 

So the functional of Eq. [65] is replaced by (some factors of 2 have been added to simplify slightly the notations) 
* = ^ - p ^[M even - 1) - p ° dd (N odd - 1) - 0( — - S ) 



53 (a/-- 4 - 1) - 53 A f - !) - E ^r 6 " ( s r en ) - E ( s r 



1/ I : Jq 



V t 



53 p even (i) 53 Ai-tj In Aj_>j + 53 P° dd (i) 53 £^ In £? iH 

-p e " en (53 p euen (i) - 1) - / 9 od<i (53^ odd («) - 1) 

i i 

-p^ 53 (i)^+E podd (^ -2^ -E A ^ (e^-*-^ -E A ? (e^-^ 
- E ^ ven I E podd (j)Bj^ - p even (i) ] - 53 v° dd I E pe " en (i)^i^ - p^ii 

i \ 3 j * \ j 

-v (j2K{i) [P odd {i) - P even (i)] - J ^J 



The optimization with respect to A^j 

(5* 







SA 



= p™™( t ) [_ hi Air+j - 1 - Xf - fif d ] 



yields 



A; 



A ,ndd 



The normalization of Eq. IB2I yields that Xf is independent of i and given by 

The stationary Equation of Eq. IB3I yields using Eq. IB4I 

P° dd U) =^2P even (i)Ai^j = C^P even {i)) 



-Pi 



(BIO) 

(Bll) 

(B12) 
(B13) 

(B14) 



The optimization with respect to -Bi-w yields similar equations and solutions, so that finally one has the simple forms 



A^ = P° dd (j) 



e ^ 



The optimization of Eq. IBlOl with respect to p even (i} yields 
<5* 







yields using Eq IBlSI 



SP even (i) 



E - 1 - •• ln - 1 ' 



^ + M ^e„ _ J2 p° dd A^ - vK{i 



even even 

H'i ~ r 



/3Ei - vK{i) + In ^53 e~^ dd ^ 



(B15) 



(B16) 



(B17) 
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Eq. IB15I then yields 



'(0 = v^rr = v , P _, M (bis) 



Similarly, the optimization with respect to p even (i) yields 



^ = £ e -,E k+vKk CB19) 

So here in contrast to the case of the homogeneous Markov chain discussed in section|Vl one obtains two Boltzmann- 
Gibbs distributions, without any prefactor like the functions {y%,Zi) in Eg 1821 This simple result was found on the 
special case of the energy flow Ki = Ei by Favretti 10] ( who has used the correct expression for the Kolmogorov-Sinai 
entropy of the alternate Markov chain, whereas an erroneous expression has been actually used in the initial work Q 
) : the even and odd stationary distributions of Eq. IB18l and lB19l are then two Boltzmann-Gibbs distributions at two 
different temperatures that are fixed by the constraints on the averaged energy and on the averaged energy flow. 

Despite the technical simplifications of the alternate Markov chain, we find that it is rather artificial from a physical 
point of view. Since the result means that the system oscillates between two distinct Boltzmann-Gibbs distributions, 
we feel that it could only correspond to physical situations where the energy is added or removed instantaneously at 
even and odd times, and that the system then relaxes to its new equilibrium during the macroscopic time r of the 
Markov chain. Otherwise, if the energy were added and removed continuously during the time intervals, this would 
mean that one should somehow have thermal equilibrium at all times with an adiabatic change of temperature, which 
seems extremely restrictive. This is why in the present paper, we have chosen to put the main focus of the case of 
the homogeneous Markov Chain. 
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